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School-based mental health services offer unparalleled opportunities for providing 
accessible care to children and adolescents. Research indicates that services available 
in schools are rarely based on evidence of effectiveness and are typically disconnected 
from the larger school context. To address these issues, the current paper presents 
initial studies to inform the development of a brief, evidence-based, flexible mental 
health intervention that fits the school context while maintaining clear structure. 
Results from two qualitative research studies - key informant interviews with school 
mental health experts and a nominal group decision-making process with stakeholders 
- are presented, both of which were aimed at informing intervention development and 
testing assumptions about how best to design an effective, context-specific mental 
health intervention that can be flexibly applied in educational settings. An explicit 
focus on educational outcomes within the context of mental health service delivery was 
identified as a key component of this integration. The paper concludes with a discussion 
of how this research has influenced the ongoing development of the intervention 
protocol, exemplifying a collaborative and iterative approach to developing school- 
based programmes. 
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Barriers to the use of research evidence in school-based mental health 

Nationally, there has been growing emphasis on increasing the availability of high-quality 
evidence-based practices (EBPs) in community-based service settings (Fixsen, Naoom, 
Blase, Friedman, & Wallace, 2005; McHugh & Barlow, 2010; Rotheram-Borus, Swendeman, 
& Chorpita, 2012). Reviews of school-based mental health (SBMH) services, however, 
suggest that the care delivered is not likely to be based on empirical research evidence (Evans 
& Weist, 2004; Rones & Hoagwood, 2000), significantly reducing the likelihood that services 
will be effective and, consequently, compromising their potential public health impact when 
delivered to youth on a large scale. Several factors unique to the education sector contribute to 
the quality and effectiveness gap (see Forman, Olin, Hoagwood, Crowe, & Saka, 2009; 
Forman et ah, 2013 for a comprehensive discussion of barriers). 

First, the policies and organizational structures represented in school programming 
may influence how and to whom mental health services are provided. For example, school- 
based initiatives such as Response to Intervention (Rtl) or Multi-Tiered System of 
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Supports (MTSS) - academic intervention models where approximately 80% of students 
are being supported by services in ‘Tier 1’ (universal), 15% in ‘Tier 2’ (selected), and 5% 
in ‘Tier 3’ (indicated) -provide early, systematic, and research-based instructional 
assistance, supported by frequent progress monitoring, to children experiencing learning 
difficulties (Barrett, Eber, & Weist, 2012; Bradley, Danielson, & Doolittle, 2007; Cheney, 
Flower, & Templeton, 2008; Gamm et al., 2012). Although these policies would seem like 
natural partners with mental health, most strategies for providing mental health care in 
schools are not integrated with Rtl/MTSS initiatives or other school structures. 
Furthermore, as described in greater detail elsewhere in this special series (Pullmann, 
Bruns, Daly, & Sander, 2013), most SBMH programmes do not explicitly promote or 
monitor academic outcomes (Atkins, Hoagwood, Kutash, & Seidman, 2010; Franklin, 
Kim, & Tripodi, 2009; Lyon, Borntrager, Nakamura, & Higa-McMillan, 2013). Second, 
many unique characteristics distinguish SBMH service providers and recipients. For 
instance, providers experience significant time constraints (Lyon et al., 2013) given the 
need to deal with frequent student crises and the sheer number of students in need of 
services. Relative to other settings, students who seek treatment in school may also 
demonstrate a broad spectrum of acuity ranging from early signs of distress to full 
psychiatric diagnoses (Lyon, Charlesworth-Attie, Vander Stoep, & McCauley, 2011; 
Walker, Kerns, Lyon, Bruns, & Cosgrove, 2010). Finally, very few of the interventions 
used in SBMH have been designed for or tested in education sector service delivery 
settings (Paternite, 2005; Wong, 2008). Instead, most of the EBPs implemented have been 
transported from other contexts. In the light of the unique system, provider, and recipient 
characteristics described above, efforts to maximize the contextual appropriateness or ‘fit’ 
of interventions to the school setting are particularly relevant to EBPs implementation in 
SBMH (Lyon et al., in press-b). 


Enhancing intervention -setting fit with a common elements approach 

In the last decade, new models of service provision have emerged which facilitate the use 
of research knowledge in practice by capitalizing on the similarities among evidence- 
based treatment protocols for youth mental health problems. This type of common 
elements approach (Chorpita, Daleiden, & Weisz, 2005a) generally allows for more active, 
flexible, and individualized delivery than traditional manualized approaches. To this end, 
recent common elements approaches make explicit use of modularized design (e.g., Weisz 
et al., 2012) in which individual components can be implemented independently or in 
complement with one another to bring about specific treatment outcomes (Chorpita, 
Daleiden, & Weisz, 2005b). Similar to Rtl and MTSS, this type of approach relies heavily 
on routine outcome monitoring - using standardized tools (e.g., symptom rating scales) 
and individualized, quantifiable targets (e.g., school attendance) - to determine whether a 
particular intervention course should be altered or maintained (Lyon, Nakamura, & Higa- 
McMillan, 2013). 

A modularized, common elements approach to psychotherapy has been found to be 
more acceptable to providers than psychotherapy based on standard treatment manuals 
(Borntrager, Chorpita, Higa-McMillan, & Weisz, 2009) and more effective than either 
standard manualized or usual care (Weisz et al., 2012). Furthermore, due to its flexibility, 
the common elements approach may be particularly appropriate for use with ethnic and 
cultural minority youth, such as those likely to be seen in schools (Lyon, Lau, McCauley, 
Vander Stoep, & Chorpita, in press-a). For these reasons, multiple efforts have been 
undertaken to evaluate the utility of the common elements approach for use by SBMH 
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providers, with some encouraging results (Lyon, et al., 2011, in press-b; Stephan, Wissow, 
& Pichler, 2010; Stephan et al., 2012; Weist et al., 2009). 


The Brief Intervention for School Clinicians 

Our research team initiated a project to develop a mental health intervention for high 
school students, called the Brief Intervention for School Clinicians (BRISC), which could 
fit optimally with the school context while simultaneously maintaining a clear structure 
and use of evidence-based intervention and assessment techniques. BRISC is designed to 
be responsive to constraints on school-based clinicians’ time, align with primary 
presenting problems of students, integrate an active focus on academic progress, and be 
useable by the varied professionals who provide mental health care in schools. BRISC was 
initially characterized by five primary elements, each intended to address specific barriers 
to effective treatment. These elements and their rationale are displayed in Figure 1, and 
discussed below: 

(1) Use of a systematic problem-solving orientation to help providers identify key 
intervention targets, test solutions, and support positive student development; 

(2) A common elements approach to delivering empirically supported mental health 
intervention components which uses specific treatment modules that match 
students’ presenting problems (e.g., mood-changing skills); 

(3) A stepped care/tiered Rtl structure that integrates BRISC into the Rtl (or MTSS) 
framework and describes how to step up or down to more or less intensive options 
(from its initial development, BRISC has been conceptualized as a Tier 2’ 
intervention); 

(4) Culturally informed treatment engagement and motivation strategies built into the 
intervention; and 

(5) Systematic assessment and monitoring, including brief, data-driven check-ins 
regarding symptoms, academic success, practice of skills, and student satisfaction. 

Building on this framework, the current paper describes two initial development 
studies designed to inform BRISC’ s iterative development process. 


School /SBHCConttxt > N'odH ► BRISC Elements 



Figure 1. BRISC logic model. 


Downloaded by [Aaron R. Lyon] at 07:12 15 November 2013 


4 


A.R. Lyon et al. 


Initial BRISC development studies 

As an initial step in the BRISC development process, two studies were designed to gather 
expert input to inform iterative intervention development in a manner that would 
demonstrate a high degree of intervention -setting fit (i.e., most likely to be practical, 
useful, and aligned with the identified needs of the education setting, students, and service 
providers). In both studies, the preliminary components of the BRISC model (above) were 
presented to participants to elicit their feedback and discussion. In Study 1, individual 
interviews were conducted with local and national key informants, all of whom are 
involved in SBMH. Informed by the results of the first study, Study 2 involved a two-day 
meeting (i.e., the BRISC ‘Development Summit’) in which participants took part in a 
structured information-generation and prioritization process (the Nominal Group 
Technique, described below). Although both studies were conducted within a single 
urban school district, they were designed to generate information that would both be 
relevant to the immediate, local initiative and generalizable to other settings in a future, 
larger-scale BRISC trial. 

Drawing from a comprehensive review of educational and mental health services 
research, we identified four key domains within which to organize participant input from 
both studies. First, we focused on the integration of mental health service delivery into 
schools, including the multi-level characteristics of the context (policy, organization, 
providers, and recipients). Second, we sought input on how to develop a treatment based 
on evidence for effectiveness as well as feasibility for implementation by SBMH clinicians. 
Third, in consideration of the populations most commonly served in schools, we collected 
information about ensuring that the intervention could engage a diverse group of students 
in a way that reflects principles of cultural and linguistic competence. Finally, we 
investigated how best to collect and incorporate relevant outcomes and data into the 
intervention, including information about mental health symptoms and an explicit focus 
on the use of indicators of academic functioning and progress. 


Study 1 (key informant interviews) 

Method 

Participants 

A total of 21 people were invited to participate in key informant interviews on the basis of 
their experience working in SBMH nationally and locally. Of the 21 invited participants, 
13 completed semi-structured interviews (62%). The key reason for non-participation was 
limited availability during the short window between Study 1 and Study 2. Six of the 
respondents were national experts in SBMH who were also members of an advisory panel 
and development workgroup for the project. Other respondents included the school 
district’s health services programme manager; one member of a mental health provider 
organization; two staff members from a public health organization in King County, 
Washington; two counsellors from school-based health centres (SBHCs); and one SBHC 
mental health supervisor. 


Procedures 

Prior to being interviewed, participants received materials summarizing the rationale for 
the project, a description of how the proposed intervention was designed to address social 
and emotional problems that interfere with academic success, the theory of change, a 
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description of the initial components of the intervention, and an overview of the interview 
questions. Phone interviews were conducted by three faculty members from the research 
group between June and August 2012. Interviews took approximately one hour to 
complete, during which the interviewer took detailed notes about participant responses. 
An exemption from full review was obtained from the University of Washington Human 
Subjects Division. 

Questions in the semi-structured interview (and the topic areas to which they were 
most relevant) included: (1) Does the intervention align with current theory, evidence, and 
real-world conditions (integration, content/structure)? (2) Which literature or policy 
statements are not referenced in the materials that might be relevant to this project 
(content/structure)? (3) To what extent will the proposed intervention structure, practices, 
and sequence be relevant and helpful for students referred to or seeking help for social- 
emotional problems in schools (integration, engagement)? (4) What changes, refinements, 
or additions to the logic model, the theory of change, or the proposed intervention 
components are needed (content/structure)? (5) How do you think this intervention could 
be made more compatible with the goals and structure of the school setting (integration)? 
(6) How could this intervention be modified to more effectively support academic 
performance goals (outcomes/data)? (7) What approach would you take to support high 
school students whose emotional health problems were impeding their academic success 
(all topic areas)? 


Analytic strategy 

Interviewer notes of key informant responses were coded by a master’s level research 
coordinator and an advanced graduate student in Public Health using Atlas. ti qualitative 
software (Muhr, 2004). A directed content analysis approach was used to code the data. In 
directed analysis (Hsieh & Shannon, 2005), a priori themes are derived from 
characteristics of the phenomena being examined, agreed upon definitions and theories 
from the literature, as well as the knowledge and experiences of researchers (Ryan & 
Bernard, 2003). As described in the introduction, the four overarching topic areas 
identified through this approach were (1) integration of mental health service delivery into 
schools , (2) content and structure based on feasibility and evidence for effectiveness , (3) 
student engagement and cultural/linguistic responsiveness, and (4) use of relevant 
outcomes and data (including school data) for monitoring and feedback. 

Coders first coded several interview transcripts independently, but then met to 
compare and consolidate their codes. All codes were categorized into one of the four 
themes. This process resulted in the creation of a codebook that was used to code the 
remaining interview transcripts. As coding proceeded, additional codes were added to the 
codebook, previous transcripts were re-coded against new codes as necessary, and coders 
checked in with one another frequently to ensure there was consensus regarding code 
assignment. Whenever there was disagreement, coders sought input from one or more of 
the research team members who conducted the initial interviews to serve as a tiebreaker. 

Results 

Integration of mental health into schools 

Rtl integration 

Of the nine (69%) respondents who made statements about the integration of the Rtl 
framework, the majority felt that the intervention would be influenced by the structure of 
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existing Tier 1 supports (i.e., supportive climate, engaged adults and students) (see 
Table 1). Others felt the project’s fit within the three-tiered model needed additional 
clarification: ‘In School Psychology, Tier 2 interventions are most often carried out with 
groups. Tier 3 kids may get ongoing counselling. This brief intervention delivered to a 
targeted group of individuals, but not particularly individualized, is somewhere in 
between’. 


Identifying students 

Nine (69%) respondents referred to strategies for identifying students to participate in the 
intervention. The majority of respondents emphasized a need for consistent school-wide 
screening and referral, via validated, standardized measures. As one respondent puts it: 

Most referrals now are self-referrals ... It would be better to start with an explicit target group, i. 
e., students with academic problems. When school people are asked to refer kids to mental 
health care, they identify kids with high-end behavioural problems and miss those with 
internalizing problems. It would be good to look for opportunities to systematically identify kids. 


Table 1. Number and percentage of Study 1 participants mentioning themes. 

N % 


Integration of mental health into schools 


Response to intervention (Rtl) integration 

9 

69 

Identifying students 

9 

69 

Engaging school staff 

6 

46 

Focus on academics 

6 

46 

Other considerations and questions 
intent and structure of intervention 

5 

38 

Clinician training 

11 

85 

Homework 

8 

62 

Treatment content 

8 

62 

Stepped care 

8 

62 

Positive features and strengths 

7 

54 

Challenges 

7 

54 

Questions from interviewees 

5 

38 

Individualization is crucial 

4 

31 

Perceived outcomes of BRISC 

3 

23 

What could be added? 

3 

23 


Student engagement and cultural linguistic responsiveness 


Type of student that can benefit from intervention 

8 

62 

Challenges 

7 

54 

Considerations of family involvement 

6 

46 

Other considerations 

2 

15 

Monitoring and feedback: use of school data 
Data for monitoring and feedback 

8 

62 

Format of feedback 

7 

54 

Who should receive feedback? 

6 

46 

Who should provide feedback? 

5 

38 

Frequency of feedback/monitoring 

5 

38 

Challenges 

5 

38 

Praised by respondents 

4 

31 

Best practices/ways to address challenges 

2 

15 

Other considerations and questions 

7 

54 


Note: Total number of participants n = 13. 
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Engaging school staff 

Six (46%) respondents made statements about engaging school staff as a method of 
supporting the intervention and its successful implementation. Some made specific 
recommendations about educating school staff, with one suggesting ‘in-service trainings 
for teachers and staff regarding particular skills such as problem-solving, psychoeduca- 
tion, communication analysis, etc.’ as well as ‘educating] them about general adolescent 
development [and] social-emotional health’ . 


Focus on academics 

Six (46%) respondents made statements about the need to focus on academic outcomes in 
the context of a school-based intervention. Some respondents expressed scepticism that 
mental health treatments, even those available in schools, would be able to meaningfully 
influence academic outcomes (‘It is not compelling to say that simply reducing anxiety 
will lead to learning because the kid is probably still not learning. The problem we have in 
mental health is that it is not its goal to improve learning’). Other respondents pointed out 
that academic and other school outcomes are not universally relevant, noting that only a 
subset of the students receiving SBMH services experience academic problems. 


Content and structure of intervention 

Clinician training 

Of the sample, 1 1 (85%) respondents gave input about training clinicians to deliver the 
intervention. The majority of these respondents believed a wide range of school-based 
professionals could be interventionists, including counsellors, school psychologists, and / 
or school nurses. As one respondent explained, ‘All nurses in [local school district] have a 
Bachelor’s degree or are nurse practitioners which means they have had mental health 
training [and] could be mental health interventionists’. Instead of focusing on particular 
types of providers, other respondents asked questions about the specific skills to be taught 
and the training process itself (e.g., ‘Clinicians really need to learn how to pick the 
components and elements they will use’). Finally, some respondents believed that 
obtaining clinician buy-in might be difficult because it deviates significantly from the 
usual clinical practice (e.g., ‘Not all clinicians will buy-in . . . How will clinicians be 
prepared for such a radical change in practice?’). 


Homework 

Eight (62%) respondents made statements about the homework assigned to students as a 
part of the intervention, with the majority of respondents believing that the success of the 
homework component may vary by student (e.g., ‘Whether students can complete 
homework will be a function of the student’s self-regulatory capacity’). Other respondents 
indicated that framing between-session practice as ‘homework’ may carry negative 
connotations for some youth and suggested that alternative wording, such as ‘practice 
exercises’, be used. 


Treatment content 

Of the eight (62%) respondents making statements about treatment content, the majority 
voiced support for the specific BRISC treatment modules we had initially selected. Similar 
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to comments about homework, respondents stated that the helpfulness of psychoeducation 
could vary by student. Some suggested that psychoeducation ‘gives the kid hope . . . that 
she/he has a problem that can be solved’. Others stated that psychoeducation can be 
effective as long as it is in relation to a student’s context, but suggested that 
psychoeducation can sometimes lead to increased reporting of symptoms (‘Psychoedu- 
cation is most effective if it builds on the student’s story and is predicated on what is 
known about the child’s background’). 

Several respondents supported selection of a limited number of modules that were 
tailored to typical student presenting problems. Finally, several respondents believed that 
it would be difficult for students to learn new skills within such a brief time frame. As one 
respondent puts it, ‘ ... it is hard to shift ownership of skill from counsellor to student in 
four sessions. Often multiple opportunities to practice are needed to really learn a skill’. 


Stepped care 

Eight (62%) respondents made statements referencing stepped care, one core feature of the 
design of the intervention. The majority of respondents wanted to know how the need for 
more intensive services would be determined after the four sessions were completed. 
Other respondents believed that the protocol provided an opportunity for improving 
service delivery (e.g., ‘I think it is a good way to organize resources and be more efficient 
with them’). 

Student engagement and cultural/linguistic responsiveness 

Regarding engagement of students, the majority of respondents made statements related to 
the type of student that would be most likely to benefit from the intervention, challenges 
surrounding student engagement, and considerations for family involvement. 


Students most likely to benefit 

Eight (62%) respondents discussed type(s) of students who could benefit from the 
intervention. Some comments suggested that students who already demonstrate high 
levels of resilience could be among the most appropriate candidates (e.g., ‘students who 
have an adequate combination of resilience, internal resources, and some family support, 
but are floundering emotionally, socially, and academically and need help tapping into 
their personal and family resources’, ‘those who have overall been doing pretty well [B or 
C student], but then something happens. There is a life stressor etc. . . . They are generally 
keeping things together, but need more support before it gets too severe’). In contrast, 
students with poor attendance and transient or situational problems were thought by some 
to be less appropriate (e.g., ‘kids with very poor attendance would not be good candidates 
unless the engagement strategies were very effective’). 


Student engagement challenges 

Seven (54%) respondents made statements about the potential challenges to student 
engagement, on the basis of developmental considerations or cultural and linguistic 
compatibility. This included asking how key therapeutic processes, such as emotional 
expression, may vary cross-culturally (‘how well do students align with the emotion that is 
selected as the target for intervention in the first session?’). A number of respondents 
underscored the importance of building initial rapport between clinicians and students and 


Downloaded by [Aaron R. Lyon] at 07:12 15 November 2013 


Advances in School Mental Health Promotion 


9 


attending carefully to adolescent development. Other respondents believed that it would 
be challenging to engage adolescents with mental health issues and maintain their 
motivation over time (‘turning the corner from brief and irregular to BRISC engagement 
will be a tough sell because it is challenging to engage adolescents in treatment and 
schedule sessions at times when they do not feel they are in an immediate crisis situation’). 


Family involvement 

Six (46%) respondents indicated it was important to build family involvement into the 
intervention. Specifically, respondents encouraged creation of a school climate where 
parental involvement is encouraged (e.g., ‘Create support, accountability, and incentives 
for clinicians to involve them. Create an individualized option for families - come in, or 
be on the speakerphone - we will send a report of our progress. This needs to happen with 
every student’). Related, respondents also highlighted the importance of being cognizant 
and considerate of family culture and values across a range of diversity issues (e.g., 
‘Paying close attention to the culture of the student [race, recent immigration, and socio- 
economic status] throughout the intervention’). 


Monitoring, feedback, and use of school data 

Data for monitoring and feedback 

Nine (62%) respondents shared opinions about the type of data they felt would be most 
important to monitor. Indicators such as attendance, grades, and motivation for change 
were cited frequently (e.g., ‘It is important to keep focus on academic progress and to 
choose discrete indicators such as attendance and tardiness’). 


Format of feedback 

Seven (54%) respondents provided suggestions related to the format of feedback. The 
majority of respondents suggested the use of an online data system, as a means of 
efficiently being able to review progress across students receiving therapy and/or with 
individual students. 


Who should receive feedback? 

Six (46%) interviewees gave their opinions about who should receive feedback. Many felt 
that feedback should be shared with an integrated team of school personnel who are 
working to support a given youth. Other respondents believed principals should receive 
feedback. In recognition of confidentiality concerns, some respondents indicated that 
students should make the determination about who is informed about their problems and 
progress. 


Data-monitoring challenges 

Five (38%) respondents referenced challenges that may be encountered with monitoring 
and feedback. Commonly cited challenges included data firewalls and privacy laws that 
govern healthcare and education, sometimes in conflicting ways. For example: 

Privacy laws prevent interaction among mental health and school staff. HIPAA makes it 
challenging for school-based health centres to share information with teachers. FERPA 
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prevents teachers from collaborating with school-based health centres. How these laws are 
interpreted seems to vary from teacher to teacher. 

Other respondents anticipated difficulty transitioning from focusing simply on health and 
mental health outcomes to an approach that monitors school outcomes as well (e.g., ‘Our 
organization was initially reluctant to steer away from tracking more clear health-related 
indicators, such as pregnancies and symptoms, and accept grades and other academic 
goals as health outcomes’). Nonetheless, several respondents expressed optimism about 
the possibilities offered by more explicitly focusing on school data. As noted by one 
respondent: 

Public health and mental health practitioners have changed their tune and acknowledge the 
value of both because of the evidence base and because they recognize school success as an 
important measure of functioning for youth, as occupational success is for adults. Although 
they initially resisted the inclusion of academic goals in contracts, they have come to see that 
helping students succeed in school is an important functional outcome. 


Study 2 (nominal group technique) 

Method 

Participants 

After the key informant interviews had been conducted, 30 local and national experts 
attended a two-day BRISC Development Summit, facilitated by project personnel, during 
which all Study 2 data were collected. Participants included researchers in various 
domains of mental health services and school mental health, Seattle Public School District 
employees, SBMH service providers and supervisors, and representatives from the local 
department of public health. Of the participants in Study 1, 11 (85%) also participated in 
the Development Summit. 


Procedure 

During the Summit, attendees participated in two rounds of small group work in which 
they engaged in a structured, consensus-based form of qualitative data collection referred 
to as the Nominal Group Technique (NGT). The purpose of NGT is to introduce an issue to 
the study group, generate information in response to the issue, and prioritize that 
information through a structured process of group discussion and voting (Potter, Gordon, 
& Hamer, 2004). 

Each NGT group included 8-10 participants and was focused on one of the four 
previously identified topic areas. There were eight total small group NGT processes, two 
on each of the four topics. Each attendee was assigned to two groups that aligned with their 
interest or expertise. In each group, participants were asked three questions related to the 
group’s identified theme: (1) what resonates or what is a strength of the BRISC 
intervention as presented?; (2) what is a concern, weakness, or something that has not been 
well addressed?; and (3) what action steps would you suggest for the project moving 
forward? Questions were the same across areas, but participants were asked to consider the 
questions as they related to their topic area. To answer each question, the groups employed 
the following five steps, consistent with the NGT model: 

(1) Opening statement: An orientation to the purpose, rules, and procedures of the 
group process. 

(2) Silent generation of ideas: Participants recorded all ideas that occurred to them. 
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(3) Round robin: Participants stated one idea at a time, with each recorded on a group 
flip chart, until all ideas were presented or a time limit was reached. 

(4) Clarification of ideas: Participants were invited to seek verbal clarification or 
further details about the ideas listed, while the facilitator kept the process as 
neutral as possible and consolidated duplicated ideas. 

(5) Voting and ranking: After clarification and consolidation, participants voted on 
priorities by placing stickers on the flip chart. Each participant was allotted three 
votes. 

For the current project, the fifth step of voting and ranking was carried out only for the 
third question (what action steps were suggested for BRISC development). Each group’s 
top five ideas were then presented to the entire group of attendees for a second 
prioritization vote. 


Results 

Results of the final voting on priority recommendations in each of the four topic areas are 
presented in Table 2 and discussed below. 


Integration of mental health into schools 

Of the participants, 60% endorsed that BRISC should focus on developing a targeted 
intervention that can be integrated within a school’s existing tiered (Rtl) system. They 
advised against building an independent, and likely redundant, multi-tier approach 
through which to respond to student mental health needs. In line with that perspective, 
53% of participants also endorsed the need to identify the appropriate oversight team 
within the school setting or establish a building-level oversight team in each school that 
would supervise the intervention’s integration with Rtl and other existing school 
structures. 

Other frequently endorsed ideas included adopting education sector language about 
positive behaviour supports when describing BRISC to facilitate buy-in and understanding 
by all stakeholders (53% of voters endorsed this). Participants supported a proposal to 
identify school-level implementation readiness criteria (which may include the 
accessibility of academic data to mental health providers) and develop a way to measure 
a school’s ability to integrate the BRISC programme (30% endorsed). The final 
recommendation for promoting integration involved the importance of working to ensure 
students have an active voice in the intervention development process and in selecting 
indicators of 'progress’ for outcome monitoring (30% endorsed). 


Content and structure of intervention 

Of the participants, 57% endorsed the recommendation that academic interventions and 
outcomes should be incorporated into the programme’s modules along with mental health 
interventions and typical mental health outcomes, suggesting a need for increased 
emphasis on academic success. This sentiment was linked to a concern that, even when a 
mental health intervention is delivered, academic difficulties may persist and, if 
unaddressed, can overshadow symptom improvements. Of the sample, 50% also felt that it 
was important to develop a detailed plan of how each aspect of the intervention would 
integrate into the clinician’s workflow as well as relevant organizational structures and 
demands. Furthermore, ensuring that flexibility and individualization are promoted in a 
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Table 2. Number and percentage of participants endorsing recommendations. 



N 

% 

Integration of mental health into schools 

Keep focused on developing a targeted intervention to exist in a 

18 

60 

tiered system 

Build an accountable oversight team 

16 

53 

Identify the appropriate ‘language’ for the intervention 

16 

53 

Identify ‘readiness’ criteria and measures 

9 

30 

Ensure active student voice in determining ‘progress’ 

9 

30 

Content and structure of intervention 

Incorporate academic interventions and monitoring academic 

17 

57 

success with students 

Plan for integration into clinician workflow and organizational 

15 

50 

structure 

Operationalize cultural responsivity within an evidence-based 

14 

47 

structure 

Articulate and plan for BRISC to include implementation 

11 

37 

preconditions 

Ensure that individualization is promoted in a structured and 

10 

33 

systematic way 

Student engagement and cultural/linguistic responsiveness 

Define a basic model where fidelity is central as well as areas that 

15 

50 

can be applied more flexibly 

Ongoing training and support for clinicians regarding culturally 

14 

47 

responsive treatment 

Think about motivational interviewing as fundamental 

13 

43 

engagement strategy rather than a tool to be integrated as needed 
Establish cultural consultation between clinician and cultural 

9 

30 

brokers for specific cultural issues 

Include cultural self-awareness in training process 

6 

20 

Monitoring and feedback: use of school data 

Create a BRISC steering committee that includes providers, 

22 

73 

school representatives, families, and students 

Use brief measures that can facilitate student identification of 

14 

47 

important outcomes 

Align goals and measurement strategies with ongoing school 

12 

40 

routines 

Make use of existing data systems and complete a data inventory 

12 

40 

Demonstrate the benefits of monitoring and feedback of data to 

9 

30 

school and provider opinion leaders 




Note: Participants in the nominal group process ( n = 30) were each allotted three votes with which to endorse 
recommendations. 


structured and systematic way that maintains fidelity to EBP was endorsed by 33% of 
respondents. 


Student engagement and cultural/linguistic responsiveness 

When providing recommendations addressing student engagement, including methods of 
ensuring that the programme would be responsive to a range of cultural groups, 
participants prioritized ideas that were centred largely on developing clinician skills and 
supporting new clinician behaviour. Similar to comments in the ‘Content and Structure’ 
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section, 50% of participants prioritized defining a basic intervention model in which 
fidelity to core components was central but which still allowed flexibility in other areas to 
meet individual student needs. Of the respondents, 47% prioritized having training 
and support for clinicians regarding cultural responsiveness. It was specifically noted that 
these efforts should be ongoing and not limited to the pre-intervention training period. 
Other priority recommendations included using Motivational Interviewing (Miller & 
Rollnick, 2002) as a fundamental engagement strategy for use with all youth, rather than a 
tool to be used only when engagement issues are identified by the clinician (43% 
endorsed), and establishing cultural consultation between the clinician and community- 
based cultural brokers for specific cultural issues relevant to students and families (30% 
endorsed). 


Monitoring and feedback: use of school data 

Related to the use of data for the purposes of monitoring and feedback, 73% of participants 
endorsed the creation of a diverse BRISC steering committee or project team - 
comprising providers, educators, parents, and students - which would consider the use of 
school data for monitoring and feedback at multiple levels. Of the participants, 47% voted 
for using brief measures in the treatment process when possible, which can 
facilitate student identification of academic and socio-emotional outcomes on which to 
focus. The Top Problems Assessment (Weisz et al., 201 1) was specifically mentioned as a 
promising tool for identifying and evaluating individualized monitoring targets. 
Participants also prioritized aligning goals and measurement strategies with ongoing 
school routines (40% endorsed) and making use of existing school district data systems by 
completing a comprehensive and district-specific data inventory (40% endorsed). Both of 
these recommendations acknowledged the potential for school and mental health data 
systems to supplement one another and were therefore identified as important avenues for 
integration. 


Discussion 

There is a growing recognition of the importance of emotional health in facilitating 
academic success (Vander Stoep, Weiss, Kuo, Cheney, & Cohen, 2003). Meanwhile, 
public resources for youth mental health treatment are shrinking (Green et al., 2013). As a 
result, the role of schools in identifying and addressing mental and behavioural issues 
among youth is likely to continue to expand. To maximize the reach and positive effects of 
SBMH programmes, it is imperative to develop and use approaches that can demonstrate 
effectiveness in promoting emotional well-being and academic success as well as a high 
degree of compatibility with the school context. The current project represents a real- 
world example of an effort to develop and ultimately test an evidence-informed mental 
health intervention that is explicitly designed to fit the structure, resources, and mission of 
schools, address the typical presenting emotional health problems of secondary school 
students, and integrate into the workflow of SBMH clinicians. To this end, three cross- 
cutting themes emerged from the two studies presented in this paper: (1) alignment with 
the school context, (2) flexible/responsive service delivery, and (3) effective data 
utilization. Furthermore, another theme related to the special issue was the consistently 
endorsed notion of increasing integration through the use of school data. These key 
findings are discussed in the context of ongoing BRISC protocol development and their 
relevance to SBMH more generally. 
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Key findings 

Alignment with the school context 

The importance of aligning BRISC with the needs, structures, individuals (providers, 
youth, and families), and resources in schools was a consistent finding across both studies. 
Indeed, education-based frameworks such as Rtl, MTSS, and School-wide Positive 
Behavior Support (SWPBS; Sugai & Horner, 2006) - as well as mental health 
frameworks, such as Systems of Care (Pires, 2002; Stroul & Friedman, 1994) - emphasize 
collaboration among families, schools, mental health, and other youth-serving systems, 
with the goal of a well-coordinated, multi-tiered continuum of effective programmes and 
services that meet the needs of all students (Barrett et al., 2012). Input from participants in 
both of our studies reinforced that, given the complex world of school programming and 
policies, simply identifying integration or collaboration as a goal does not provide a clear 
path to its achievement. Participants in both studies recommended that we clearly specify 
where BRISC is situated within the three-tiered model of stepped service provision and 
develop guidelines for how and when to refer students to more or less intensive supports 
based on their response to this intervention. As a result, we have developed a graphic to 
describe how BRISC fits within this framework and define key inputs from, and outputs 
into, the school context at multiple levels (see Figure 2). 

Participants also referenced the need for a systematic, holistic strategy for integrating 
BRISC into individual schools. Given that school contexts vary greatly, even within 
individual school systems, several participants recommended that we develop an explicit 
implementation framework paired with a readiness assessment. Before implementing 
BRISC, a clinician or mental health implementation team can use this information to 
better understand the level of development of mental health and other supports in the 
school, facilitate connections between the clinician and other types of school-based social 
and behavioural supports, and develop an action plan around each implementation support 
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Figure 2. Framework for BRISC integration into the three-tiered model of school-based service 
delivery. 
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element. Low levels of readiness could be addressed with additional supports prior to, or in 
the process of, BRISC introduction. Specific indicators of readiness identified by advisors 
included: (1) presence of a team responsible for school-wide emotional and behavioural 
support planning (including BRISC), (2) systematic behavioural screening and assessment 
occurring in the school, (3) efforts at family engagement and outreach including events 
and parent-support groups, and (4) education and training for teachers and staff on mental 
health. Completing a comprehensive inventory of other Tier 1-2 and intensive Tier 3 
services and supports is also critical, as is an evaluation of the availability and accessibility 
of educational data to monitor student functioning in the school context. 

Another key component of school alignment, given the personnel working in schools, 
is the question of ‘who will implement BRISC?’ Study participants consistently noted that 
professionals in a range of roles could be trained and supported to effectively implement 
the intervention. Research shows that the diverse SBMH workforce (including guidance 
counsellors, school psychologists, school nurses, school social workers, and external 
agency-based clinicians) is responsible for a wide range of client care, but that the 
presence of such professionals does not necessarily mean increased mental health service 
use (Green et al., 2013; Kelly & Lueck, 2011). The brief, manualized, tiered format of 
BRISC could provide a means of engaging multiple types of professionals to provide 
mental health care (Atkins et al., 2010), thus increasing the likelihood that effective mental 
health interventions are available and used in schools. 

Effectively aligning BRISC with the school context includes careful attention to the 
students who are most likely to receive SBMH services. Previous research has identified 
that youth from ethnic and cultural minority backgrounds are particularly likely to access 
SBMH care over services in other sectors (e.g., Kataoka, Stein, Nadeem, & Wong, 2007; 
Lyon, Ludwig, Vander Stoep, Gudmundsen, & McCauley, 2013). Participant responses 
across studies indicated that there may be opportunities to design the BRISC intervention 
and training protocols to encourage cultural responsiveness directly. Evidence from the 
field of youth mental health suggests that interventions that have been explicitly culturally 
adapted, on average, may be no more effective than non-adapted programmes (Huey, 
2012); suggesting that individual clinician cultural competency may be an equally 
important area of emphasis. For these reasons, it will be important for the BRISC readiness 
assessment mentioned earlier to include specific questions about the state of cultural 
competence/competence training for school-based professionals, and for the BRISC 
programme to provide additional training as needed. 


Flexible/responsive service delivery 

The theme of flexible service delivery in schools emerged when participants were discussing 
the overarching structure of BRISC and the stepped-care model for which it is intended, as 
well as methods of tailoring service delivery to be responsive to the diverse youth seen in 
schools. Consistent with emerging models of flexible, evidence-based psychotherapy 
interventions - which have been identified as particularly applicable to ethnic and cultural 
minority youth (Lyon et al., in press-a) - the ability of a BRISC provider to select from a set 
of specific practices or skills to individualize service delivery represents an additional level 
of BRISC cultural responsiveness and engagement. This level is largely dependent on an 
individual provider’s ability to administer the BRISC protocol in a way that is appropriately 
tailored to students from a variety of backgrounds. Other input focused on the need to 
combine rigor with flexibility and simplicity, to achieve the goal of maximum applicability 
to student needs. These responses largely parallel the continuing ‘fidelity versus flexibility 
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debate’ that has permeated the literature on psychosocial mental health interventions (Cohen 
et al., 2008; Kendall & Beidas, 2007). Several advisors also recommended keeping the 
overall number of modules small, to narrow the focus of the intervention and facilitate 
practitioner learning and model adherence. Achieving a balance between simplicity and 
maximum applicability within a flexible, stepped-care model will be an important tension 
for the BRISC development team to resolve. 


Effective data utilization 

Education data (e.g., attendance, homework completion, grades) have been identified as key 
drivers of effective, multi-level data-driven decision-making in SBMH (Lyon, Nakamura, 
& Higa-McMillan, 2013). This type of information can function as a practical cornerstone 
for the integration of mental health services into schools, enhancing collaboration among 
the diverse professionals working in schools around the shared objective of improving 
educational functioning. Indeed, the importance of outcome data in general - and 
educational data in particular - was a key theme across the two studies described above and 
across all four topic areas. Some respondents indicated that educational data could be used 
to determine which students would be appropriate for the BRISC intervention (e.g., 
excluding those who are already exhibiting very low attendance or academic failure and 
tracking them into alternative, more intensive services). Furthermore, it was suggested that 
the accessibility of educational data for mental health service delivery purposes represents 
an important readiness criterion for BRISC implementation. 

Despite widespread interest in educational outcomes, participants were somewhat 
divided about the degree to which academic functioning should be a central focus of 
SBMH treatment. Some participants indicated that the primacy of the school mission and 
co-location of the practitioner within the school makes direct intervention focused on 
academics appropriate. For others, however, this appeared to represent a conflation of 
roles; these participants suggested that a more traditional facilitation of academic progress 
through removal of social-emotional barriers to learning should define the SBMH mission, 
as well as the BRISC practice model. BRISC has been conceptualized as a programme 
with the ability to improve academic and behavioural outcomes in addition to other 
traditional mental health outcomes, but it would be beyond the scope of a mental health 
service to focus on academic needs that are in the purview of educators (e.g., to enhance 
reading skills). As described in the ‘Implications’ section below, the revised BRISC 
protocol will attend to educational outcomes, but remain focused on problem-solving 
surrounding social, emotional, and practical barriers to academic success. 

Finally, respondents also discussed the ways to ‘source’ information about important 
outcomes. Integrated data systems that can identify students in need of services and allow 
for tracking multiple types of outcomes or functional domains (including mental health 
and educational outcomes) may facilitate problem-solving with students about the focus 
of treatment, monitoring progress over time, and communicating with school staff. 
To support these activities, new data systems may be created or existing data systems 
be re-purposed (Lyon, Nakamura, & Higa-McMillan, 2013). 


Limitations 

Although the current studies were formative by design, several limitations should be 
considered. First, our sample sizes were small and not fully representative of all types of 
school and mental health roles that could provide meaningful input about the development 
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of an SBMH intervention. The first study in particular only had 13 participants, and only 
one of them was an individual from the local school district (in contrast, the second study 
had considerably greater local representation). Nevertheless, approximately half of the 
participants in both studies routinely worked in the district. All participants were selected 
by the research team based on their national and local reputations, which is also not likely 
to yield fully generalizable opinions. Regarding our qualitative methodology, the research 
team identified a set of four common themes from the initial study and used these to solicit 
input from the larger nominal group process in Study 2. This may have limited the breadth 
and diversity of input we received. Finally, although the NGT provided a systematic 
method of prioritizing participant ideas (through voting), it could have biased our 
conclusions by giving greater weight to ideas suggested by persuasive group members and 
de-emphasizing ideas that did not fit the existing project narrative or contemporary views 
related to SBMH. 


Implications and next steps 

Despite the above limitations, the information gleaned from these two studies has been 
instrumental to the design and iterative evaluation of BRISC and its implementation 
supports, and may help guide other SBMH development efforts focused on integration into 
the school context. As a result of this input, we have developed an implementation 
framework that is not only specific to BRISC but also draws from SWPBS and Rtl 
assessment tools, such as the Self-Assessment and Program Review (Walker & Cheney, 
2012). Specific revisions to the intervention protocol, based on lessons learned from the 
current studies, include the incorporation of explicit evaluation of academic functioning at 
the outset (using a brief academic and functional review tool), use of a ‘Top Problems’ 
(Weisz et al., 2012) approach to identify academic and social-emotional targets, and steps 
to assure that student ‘voice’ is central in developing goals and targets. The most recent 
version of the BRISC protocol also includes a focus on problem-solving directly related to 
homework completion and school attendance as a primary component. We are currently 
working on ways to allow for easy access to school data so that the work of the mental 
health providers can be accurately geared to support student’s academic progress. Finally, 
to address the issue of a school’s ‘readiness’ to move towards a more integrated mental 
health- academic supports approach, we have begun to meet with school personnel to 
identify criteria to be incorporated into a school readiness assessment tool. 

A particularly important suggestion that emerged from both the studies was that 
BRISC’ s use of mental health and educational data for progress monitoring may be 
enhanced by support from a computerized measurement feedback system (MFS; Bickman, 
2008) designed to track youth outcomes. These types of systems and related technologies 
have the potential to advance assessment and facilitate quality improvement in schools in a 
manner that is contextually appropriate (Lyon, Nakamura, & Higa-McMillan, 2013). 
Fortunately, a parallel but independent initiative to implement a MFS for use by SBMH 
providers is currently underway in the district in which BRISC is being developed (Lyon, 
Knaster Wasse, Ludwig, Pullmann, & McCauley, 2013). 

In conclusion, as we attempt to develop a brief, effective school mental health 
intervention, our team is cognizant that ‘effective programs rarely emerge without 
consideration of their implementation contexts’ (Cappella, Reinke, & Hoagwood, 2011, 
p. 460). Principal among the contextual factors relevant to intervention effectiveness 
in schools is the systematic collection and appropriate use of educational data. 
In collaboration with school and mental health stakeholders, BRISC is being developed 
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with an explicit focus on these kinds of outcomes to enhance school-mental health 
integration and maximize intervention -setting fit. 
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